Systems and methods for providing augmented reality emergency response solutions

ABSTRACT

An emergency response system (ERS) configured to acquire target terminal location data, emergency responder terminal location data, and equipment location data, engage a camera component of the emergency responder terminal to capture images of at least a portion of the surrounding real-world scene, providing such images for display on the touchscreen display of the emergency responder terminal, and providing one or more menu items associated with a selected target terminal display object, the one or more menu items configured to activate a resource of the target terminal of the selected target terminal display object to assist the emergency responder to locate the target terminal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of, and claims thebenefit of, U.S. patent application Ser. No. 15/825,094, filed on Nov.28, 2017, entitled SYSTEMS AND METHODS FOR PROVIDING AUGMENTED REALITYEMERGENCY RESPONSE SOLUTIONS, which is hereby incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to emergency response tools,and some embodiments relate more particularly to systems and methods forproviding augmented reality emergency response solutions to aidemergency responders in providing timely and effective assistance toindividuals in emergency situations.

BACKGROUND

In emergency situations it is often difficult for emergency respondersto efficiently locate persons in need of assistance, and to renderappropriate aid to such persons. In general, there is a lack ofinformation available to such responders when they need it most. In suchsituations, it can also be difficult for emergency responseorganizations to coordinate efforts to dispatch individual emergencyresponders, or groups of emergency responders, in an efficient andeffective manner to assist the organization's members. The presentdisclosure addresses the deficiencies of conventional systems, andprovides augmented reality based emergency response systems that aidemergency responders in: identifying locations of individuals in need ofassistance (including real-time or near real-time location data);identifying path information showing such individual's trail to arrivingat their present location; identifying locations of equipment, tools, orother resources that may be useful for rendering aid in a givensituation; establishing communications channels between emergencyresponders and persons in need of assistance; summoning additionalassistance among users, emergency responders, emergency responseentities, etc.; and various other features that enable more timely andeffective assistance to persons in emergency situations.

BRIEF DESCRIPTION OF THE DRAWINGS

The technology disclosed herein, in accordance with one or more variousembodiments, is described in detail with reference to the followingfigures. The drawings are provided for purposes of illustration only andmerely depict typical or example embodiments of the disclosedtechnology. These drawings are provided to facilitate the reader'sunderstanding of the disclosed technology and shall not be consideredlimiting of the breadth, scope, or applicability thereof. It should benoted that for clarity and ease of illustration these drawings are notnecessarily made to scale.

Some of the figures included herein illustrate various embodiments ofthe disclosed technology from different viewing angles. Although theaccompanying descriptive text may refer to such views as “top,” “front,”“back,” “bottom” or “side” views, such references are merely descriptiveand do not imply or require that the disclosed technology be implementedor used in a particular spatial orientation unless explicitly statedotherwise.

FIG. 1 is a block diagram illustrating an example emergency responsemanagement system in accordance with one or more embodiments of thepresent disclosure.

FIG. 2 is a block diagram illustrating an example system emergencymanagement entity in accordance with one or more embodiments of thepresent disclosure.

FIG. 3A depicts an example emergency responder utilizing the presentlydisclosed technology in an example environment in which the presentlydisclosed technology may be implemented.

FIG. 3B illustrates an example augmented reality presentation that maybe displayed on the emergency responder terminal of the emergencyresponder shown in FIG. 3A, here showing various user locations inaccordance with one or more embodiments of the present disclosure.

FIG. 3C illustrates an example augmented reality presentation that maybe displayed on the emergency responder terminal of the emergencyresponder shown in FIG. 3A, here showing example registered user dataand related options associated with one of the target users inaccordance with one or more embodiments of the present disclosure.

FIG. 3D illustrates an example augmented reality presentation that maybe displayed on the emergency responder terminal of the emergencyresponder shown in FIG. 3A, here showing example route informationassociated with one of the users in accordance with one or moreembodiments of the present disclosure.

FIG. 3E illustrates an example augmented reality presentation that maybe displayed on the emergency responder terminal of the emergencyresponder shown in FIG. 3A, here showing example user path informationassociated with the given user in addition to the route informationshown in FIG. 3D, in accordance with one or more embodiments of thepresent disclosure.

FIG. 3F illustrates an example augmented reality presentation that maybe displayed on the emergency responder terminal of the emergencyresponder shown in FIG. 3A, here showing example user path informationassociated with multiple users in addition to the route informationshown in FIG. 3D, in accordance with one or more embodiments of thepresent disclosure.

FIG. 3G illustrates an example augmented reality presentation that maybe displayed on the emergency responder terminal of the emergencyresponder shown in FIG. 3A, here showing example equipment locations inaddition to the information shown in FIG. 3F, in accordance with one ormore embodiments of the present disclosure.

FIG. 4 illustrates another example augmented reality presentation thatmay be displayed on an emergency responder's emergency responderterminal in accordance with one or more embodiments of the presentdisclosure.

FIG. 5 is a flow diagram illustrating an example method in accordancewith one or more embodiments of the present disclosure.

FIG. 6 is a flow diagram illustrating an example method in accordancewith one or more embodiments of the present disclosure.

FIG. 7 illustrates an example computing component that may be used inimplementing various features of embodiments of the disclosedtechnology.

The figures are not intended to be exhaustive or to limit the disclosedtechnology to the precise form disclosed, and that the disclosedtechnology be limited only by the claims and the equivalents thereof.

DETAILED DESCRIPTION

Embodiments of the present disclosure provide augmented reality basedemergency response systems that aid emergency responders in identifyingreal-time or near real-time locations of individuals in need ofassistance, and in providing timely and effective assistance to suchindividuals.

FIG. 1 is a block diagram illustrating an example emergency responsesystem in accordance with one or more embodiments of the disclosedtechnology. As shown, emergency response system 1000 (hereafter, system1000) may include one or more emergency responder terminals 1100 and oneor more target terminals 1200 in communication with an emergencymanagement entity 1300 (hereinafter “EME 1300”). EME 1300 and emergencyresponder terminals 1100 may further be in communication with respondervicinity components 1400, and other components. EME 1300 and targetterminals 1200 may further be in communication with emergency vicinitycomponents 1500, and other components. As shown, the communicationbetween one or more of: emergency responder terminals 1000, targetterminals 1200, emergency management entity 1300, responder vicinitycomponents 1400, and emergency vicinity components 1500 may occur,directly or indirectly, over any one or more communications links (e.g.,wired or wireless connections) including via one or more networks 1600(e.g., cellular network, Bluetooth® network, ZigBee® network, Wi-Fi®network, etc.), inclusive of the hardware and software required toestablish such a communications link (e.g., communications interfacessuch as cellular chipsets, Bluetooth® modules, ZigBee® modules, Wi-Fimodules, etc.).

System 1000 may obtain location information of one or more targetterminals 1200 within (or associated with) a zone of interest; obtainlocation information of one or more emergency responder terminals 1100responsible for the zone of interest; provide location information ofone or more target terminals 1200 to the one or more emergency responderterminals 1100; and/or present an augmented reality display object onthe display of the emergency responder terminals 1100, the augmentedreality display object overlaying one or more regions of one or moreimages representing a zone of interest (or portion of the zone ofinterest), the regions overlaid by one or more augmented reality objectscorresponding to respective target terminal 1200 locations in thereal-world scene being captured by the emergency responder terminal1100's camera component.

In some embodiments, system 1000 may obtain location information of oneor more pieces of equipment within (or associated with) the zone ofinterest; provide location information of one or more pieces ofequipment to the one or more emergency responder terminals 1100; presentan augmented reality display object on the display of the emergencyresponder terminals 1100, the augmented reality display objectoverlaying one or more regions of one or more images representing a zoneof interest (or portion of the zone of interest), the regions overlaidby one or more augmented reality objects corresponding to respectivelocations of the one or more pieces of equipment within the real-worldscene being captured by the emergency responder terminal 1100's cameracomponent.

As shown in FIG. 1, emergency responder terminals 1100 and targetterminals 1200 may be mobile computing devices such as, for example,smartphones (as shown in FIG. 1), tablets, netbooks, laptop computers,or any other mobile terminal able to communicate over a wired orwireless network. EME 1300 may be a computing device such as a server, adatabase, a computer, a workstation, a computing terminal, a mobiletelecommunications device, an electronic storage device, a computingplatform, or other computing resource. Emergency responder terminals1000, target terminals 1200, and EME 1300 may include one or morecomputing resources available to, and in some embodiments consideredpart of, system 1000. The various features and technology disclosedherein may be effectuated by one or more of the computing resources ofsystem 1000, whether by the computing resources of a particular device(e.g., an emergency responder terminal), or by the computing resourcesof several devices performing operations in concert with one anothersuch as may the case in a distributed computing arrangement among thecomputing resources of multiple devices (duplicative, parallel, ordistributed computing of both an emergency management entity and anemergency responder terminal function or feature). Some such computingresources are shown in FIG. 2 discussed herein.

Referring still to FIG. 1, emergency responder terminals 1100 and/ortarget terminals 1200 may run an emergency management application(hereafter “Emergency App”), though in some embodiments it may not benecessary for the target terminals 1200 to run a mobile application tobe operable within system 1000. In some embodiments, Emergency App runson EME 1300 as a virtual resource accessible to emergency responderterminals 1100 and/or target terminals 1200. Emergency App mayfacilitate access to one or more resources of the emergency responderterminals 1100, target terminals 1200, EME 1300, Responder VicinityComponents 1400, Emergency Vicinity Components 1500, and may provide,distribute, or otherwise convey data obtained via any one or more ofsuch resources to one or more others of such resources.

In some embodiments, the Emergency App provides a GUI to displayinformation to the user of a terminal, and may be further adapted toreceive user input that may be conveyed throughout system 1000,including for example, to EME 1300, to other terminals, to respondervicinity components 1400, to emergency vicinity components 1500, etc. Asone of ordinary skill in the art will appreciate, EME 1300 may includeone or more APIs providing a software interface between terminals andthe resources of EME 300. In some embodiments, Emergency App may utilizeor access telephony, SMS, camera, voice, data and other resources ofemergency responder terminals 1100, target terminals 1200, respondervicinity components 1400, and/or emergency vicinity components 1500. Forexample, an emergency responder may select an icon on the GUI of theEmergency App to open a two-way communication channel between theiremergency responder terminal 1100 and another terminal such as aparticular target terminal 1200 or group of target terminals 1200, oranother emergency responder terminal 1100 such as the emergencyresponder terminal of an emergency response unit's captain/supervisor.Such communications may include SMS text messaging conversations, videomessaging, Picture messaging, in-app messaging conversations, a voicecall, a video conference, etc.

In another example, an emergency responder may select an icon or menuitem on the GUI of the Emergency App to activate a resource of a targetterminal 1200 to assist the emergency responder in locating the targetterminal 1200. For instance, an emergency responder may activate aspeaker of a target terminal to propagate an alarm sound to help theemergency responder find the target terminal (and ultimately the userassociated with the target terminal) as they near the locationidentified by system 1000. Likewise, and emergency responder mayactivate a light source (e.g., an LED) of the target terminal topropagate a flashing light to help the emergency responder find thetarget terminal (and ultimately the user associate with the targetterminal).

FIG. 2 is a diagram illustrating example resources of system 1000 inaccordance with one or more embodiments of the disclosed technology,resources that are not explicitly shown in FIG. 1. As shown systemresources 1700 may include one or more processing engines (representedsymbolically by processor 1702), one or more memories (representedsymbolically by memory 1703), and one or more electronic storage units(represented symbolically by electronic storage 1730). Memory 1703 maybe configured with machine readable instructions which, when executed byprocessor 1702, cause system 1000 to effectuate one or more featuresdiscussed below with respect to elements identified by numerals1704-1724 (which may be embodied in or effectuated by one or more ofemergency responder terminals 1100, target terminals 1200, respondervicinity components 1400, emergency management entities 1300, emergencyvicinity components 1500). System resources 1700 may include one or moreof a user location component 1704, camera component 1706, display objectcomponent 1708, registered user detail component 1710, situationalstatus component 1712, priority component 1714, mission assignmentcomponent 1716, mission route component 1718, an equipment locationcomponent 1720, communication component 1722, and other components 1724.

A user location component 1704 in accordance with one or moreembodiments of the present disclosure obtains location data of one ormore emergency responder terminals 1100 and target terminals 1200. Insome embodiments such data may be obtained via location resources (e.g.,location circuitry) local to such terminals, and may be provided tosystem 1000 over network 1600. User location data is indicative of thegeospatial location of one or more of the emergency responder terminals1100 and target terminals 1200 associated with or connected to system1000 (collectively, “units”).

User location component 1704 may include components included interminals (e.g., terminal local resources), components included inemergency management entity (e.g., communications modules,machine-readable instructions), or both. In accordance with one or moreembodiments of the present disclosure, system 1000 may obtain locationdata by actively transmitting a location data request to one or more ofthe units, thereby prompting the one or more units to access locallocation resources (e.g., GPS modules, altimeter, barometer, beacons,etc.) to obtain location information and generate and transmit a datapacket containing the location data (e.g., location details) to acomputing resource of system 1000. For instance, EME 1300 of system 1000may receive a responsive transmission from the one or more unitscontaining the one or more units' respective location data, and system1000 may propagate and/or store such information within an electronicstorage or memory of system 1000 as desired for a particular applicationor by a particular resource coupled to or within system 1000.

Location data may indicate geospatial location of a user associated withthe unit, including longitude and latitude coordinates,degrees/minutes/seconds location parameters, altitude above sea level,altitude above ground level, etc. User location component 1704 may beutilized to identify geospatial location of a user. User locationcomponent 1704 may comprise one or more circuits, modules, or chipslocal to the units themselves. For example, location component 1704 mayinclude a GPS sensor, an altimeter, a pressure sensor (e.g., abarometer), and the like. In some embodiments user location component1704 may further comprise hardware and software operating on EME 1300and communicatively coupled with location sensors of one or more units.

Location data may also indicate structure specific location details of auser, including location within or upon a structure whose dimensions orother parameters are known to or derivable by system 1000. For example,if a large office building implements system 1000 for emergencysituations, the building operator may choose to install one or moreemergency vicinity components 1500 throughout the building to enhancelocation data provided to system 1000. In some embodiments emergencyvicinity components 1500 provide location data as an alternative to, oras an enhancement to, the location data obtained by the locationcomponentry local to a unit. Emergency vicinity components 1500 mayinclude any one or more of: location beacons, proximity sensors,receivers, etc. installed throughout the building structure (on eachfloor, for example), that may provide structure specific locationdetails that may be relevant to a particular application.

For example, extending the example of an office building implementingsystem 1000, suppose the building operator chooses to install locationbeacons or receivers throughout each of its twenty floors. The beaconsmay transmit and/or receive signals to/from units within a predefinedvicinity of the beacon's location (e.g., within 20 feet of the beacon,for example), and may provide location component 1704 of system 1000with information about which units are within its vicinity. Individualbeacons reporting to system resources 1800 may be associated with aparticular location in the building (e.g., beacon may be associated withconference room D on the north side of the building on the 6^(th)floor). Such information may be pre-registered or otherwise provided inadvance to system resources 1800 as building data 1734 or electronicstorage 1730). Thus, if a registered user, Clayton, has his smartphone(target terminal 1200) with him and both were located in conference roomD on the north side of the building on the 6th floor when an emergencysituation arose (e.g., a fire in the building), user location component1704 may obtain an indication, directly or indirectly, from the locationbeacon that Clayton was located on the 6^(th) floor on the north side ofthe building in conference room D. As much information as desired abouta building or other structure, or about any of the emergency vicinitycomponents 1500, may be registered or otherwise entered and stored asbuilding data 1734 on system resources (e.g., electronic storage 1730,which may reside for example at the EME 1300 that is operably connectedto emergency responder terminals and/or target terminals). This way,location information available to system 1000 may be enhanced with asmuch detail as desired for a given application or as desired by a givenoperation.

In still a further example, and as discussed further hereafter withrespect to equipment location component 1722, in some embodimentsemergency vicinity components 1500 include location beacons, proximitysensors, receivers, RFID tags, etc. installed in an indoor locationwhere a piece of usable equipment is stowed within the building, and/oran outdoor location where a piece of usable equipment is kept. Suchequipment may include a fire extinguisher, an epinephrine pen (EpiPen),an axe, an oxygen mask, a defibrillator, a hose, a fire alarm, a firehydrant, a hose, a rope, etc. In some instances, such location beacons,proximity sensors, receivers, RFID tags, etc. may be affixed to the itemof equipment itself (or the housing or casing associated with suchequipment). In some instances, such emergency vicinity componentsassociated with a piece of equipment may include hardware, such as aspeaker or light source, that may aid an emergency responder or otheruser (e.g., a user associated with a target terminal) in locating thepiece of equipment.

For example, an emergency responder may select an icon or menu item onthe GUI of the Emergency App running on their terminal to activate,directly or indirectly, a resource of an emergency vicinity componentsuch as a speaker or light source coupled to a Bluetooth low energyproximity beacon tied to a desired piece of equipment, for example, toassist the emergency responder in locating the desired piece ofequipment in a given building. The speaker may propagate an alarm sound,or the light source may propagate a flashing light, for example, to helpthe emergency responder find the desired equipment.

Although features herein are often discussed with respect to one or theother of responder vicinity components 1400 and emergency vicinitycomponents 1500, the same or similar features discussed with respect toone may be implemented with respect to the other.

Referring still to FIGS. 1 and 2, Emergency App may utilize or operateone or more resources of system 1000, including resources of emergencyresponder terminals 1100, target terminals 1200, responder vicinitycomponents 1400, EME 1300, and/or emergency vicinity components 1500.For example, Emergency App may utilize or operate a camera component1706 of an emergency responder terminal 1100 or target terminal 1200 tocapture a portion of a real-world scene within the field of view of thecamera lens and CMOS array of the respective device.

Display object component 1708 may define display objects associated withtarget terminals 1200 and/or emergency responder terminals 1100 whoselocation data has been obtained by system 1000 via one or more systemresources 1800. A display object may include any graphic (e.g., a pin, abubble, a tile) that may be displayed on an electronic display. Displayobject component 1708 may determine when a portion of a real-world scenecaptured by a camera component 1706 of an emergency responder terminal1100 captures a region of space wherein a target terminal 1200 and/orother emergency responder terminals 1100 are located. As one or moreportions of the real-world scene captured is provided on the display ofthe emergency responder terminal 1100 (e.g., as a user scans the cameraover a scene), display object component 1708 may cause the displayobjects corresponding to target terminals 1200 and/or other emergencyresponder terminals 1100 falling within the scene to be displayed suchthat they overlay the real-world scene images in a manner (e.g., on aregion of the image) that identifies the location of respective targetterminals 1200 and/or other emergency responder terminals 1100 withinthe real-world scene as it is displayed on the display of the givenemergency responder terminal 1100.

In some embodiments, display object component 1708 defines a displayobject associated with emergency responder terminals 1100 whose locationdata has been obtained by system 1000 via one or more system resources1800. A display object may include any graphic (e.g., a pin, a bubble, atile) that may be displayed on an electronic display. Display objectcomponent 1708 may determine when a portion of a real-world scenecaptured by a camera component 1706 of a target terminal 1200 captures aregion of space wherein an emergency responder terminal 1100 and/orother target terminals 1200 is located. As one or more portions of thereal-world scene captured is provided on the display of the targetterminal 1100 (e.g., as a user scans the camera over a scene), displayobject component 1708 may cause the display objects corresponding toemergency responder terminals 1100 and/or other target terminals 1200falling within the scene to be displayed such that they overlay thereal-world scene images in a manner (e.g., on a region of the image)that identifies the location of respective emergency responder terminals1100 and/or other target terminals 1200 within the real-world scene asit is displayed on the display of the given target terminal 1200.

Registered user detail component 1710 obtains information stored in auser profile for one or more registered users. Such information may beprovided by a user when first creating a user profile upon registeringwith system 1000 (e.g., registration and sign-up via Emergency App).Such information may include the user's name, age, gender, height,weight, contact information (phone, email, etc.), contact informationpreferences, emergency contact information, family information (spouse,children, siblings, parents, etc.), employment information, skillset,completed emergency response trainings (e.g., CPR certified, tacticaltraining, bomb deactivation training), health conditions or tendencies(e.g., diabetes, asthma, claustrophobia, etc.), physical disabilities(e.g., visually impaired, hearing impaired, paraplegic, etc). Suchinformation may also include a categorical designation. The categoricaldesignation may be as simple as selecting either an “emergencyresponder” (e.g., responder) category or a “standard user” category(e.g., potential target). Such information may further include detailsabout the device the user designates as their primary terminal—e.g.,emergency responder terminal, target terminal. Any and all suchinformation may be stored in any electronic storage available to system1000, such as in a registered user data 1732 sector of electronicstorage 1730.

In some embodiments, emergency responder terminals 1100 and targetterminals 1200 are distinguished from one another by the category ofregistered users with which they are associated as stored in the userprofiles for the one or more registered users. According to someembodiments, emergency responder terminals 1100 are associated with acategory of registered users designated as emergency responders (e.g.,persons or other entities charged with responding to an emergencysituation, resolving an emergency situation, or assisting others who areinvolved in an emergency situation). According to some embodiments,target terminals 1200 are associated with a category of potential target(e.g., a standard user of or subscriber to system 1000).

Emergency responders may include public individuals, groups, orentities. For instance, public emergency responders might include: apolice department, a division of a police department (e.g., a taskforce, bomb squad, etc.), an individual police officer or group ofpolice officers; a fire department, a division of a fire department, anindividual fireman or group of firemen; a federal law enforcement agency(FBI, CIA, etc.), a division of a federal law enforcement agency, or anindividual field officer or team of field officers; a local poisoncontrol department, a division of a local poison control department, anindividual poison control agent or group of agents; or any other publicentity, individual, or group of individuals designated as such, and thelike. Emergency responders may also include private individuals, groups,or entities. For instance, private emergency responders might includesecurity guards, property patrolmen, or any other private entity,individual, or group of individuals designated as such, and the like.

In some embodiments, the emergency responders category may be furtherdivided into subcategories that have different privileges, permissions,or capabilities to interact with and control one or more aspects ofsystem 1000. In some embodiments the subcategories may correspond to thehierarchical structure relevant to a group of emergency responders. Forexample, the emergency responders in a particular scenario might includea local police task force. The task force might be made up of onecommander, one managing officer that reports to the commander, and fivefield officers that report to the managing officer. In such a scenario,subcategories might include: “commanders,” “managing officers,” and“field officers,” with descending privileges, permissions, orcapabilities to interact with and control one or more aspects of system1000. Various applications will be made apparent upon review of theentire disclosure herein.

According to some embodiments, target terminals 1200 are associated withany non-emergency responder category of registered users, which mayinclude but not be limited to registered users or entities that are indanger, are involved in an emergency situation, or which are otherwisein need of assistance from an emergency responder. Just as emergencyresponders may be further divided into subcategories that have differentprivileges, permissions, or capabilities to interact with and controlone or more aspects of system 1000, so to can the non-emergencyresponder category. For instance, if a school campus were to adopt anembodiment of system 1000 and the school campus were to be underlockdown because of a bomb threat, the subcategories of registered userswithin the non-emergency responder category may include the school'sdean, administrators, teachers, students, and visitors. Each may begiven different privileges, permissions, or capabilities to interactwith and control one or more aspects of system 1000. Variousapplications will become apparent upon review of the entire disclosureherein.

Situational status component 1712 obtains situational status informationin real-time from users in emergency situations. Situational statusinformation may include any information that provides additional contextabout the nature of the situation within the vicinity of a givenuser—e.g., number of other people in the same vicinity as the userhaving the target terminal, an injury sustained by the user having thetarget terminal (or a person in the vicinity), a reaction beingexperienced by the user of the target terminal (or a person in thevicinity), a request for specific type of help needed or number ofresponders needed, and/or images, sounds, or video of the vicinity.

Such situational status information may be provided as preconfiguredmessages/data the user can automatically send by tapping or otherwiseselecting an associated buttons, icon or tile from their Emergency App,for example, or may be customized messages/data the user types orrecords into their unit via their Emergency App, for example. Examplesof buttons, icons, or tiles that may be useful to transmit preconfiguredmessages via the target terminal version of the app may include one ormore of the following: “CPR Needed,” or “Immobilized,” or “Children inVicinity,” “Move on to other targets,” and the like. Example customizedmessages the target terminals may transmit may include messages typed bythe user of the unit, such as: “There is a person in my vicinity in needof CPR, please send CPR certified emergency responder as soon aspossible,” or “I am in the vicinity of the emergency but there is aclear path for my escape, no need to assist me, move on to others,” or“I am trapped and cannot find an exit,” or, “I've injured my leg and amimmobile, please send two or more emergency responders to carry me out,”or “the smoke from the fire is becoming unbearable, people around me arelosing consciousness and I expect I will shortly too,” and the like.

Examples of buttons, icons, or tiles that may be useful for theemergency responder terminal version of the app may include one or moreof the following: “Send additional responders,” or “Mission AssignmentCompleted,” or “Available for New Mission Assignment,” and the like.Example customized messages the emergency responder terminals maytransmit may include messages typed by the user of the unit, such as: “Ineed three more emergency responders at my present location,” or “Theperson I came to assist was passed out when I arrived, please sendanother responder with medical training within the next 10 minutes,” or“the south exit of the building is blocked by a pillar that has fallenin front of the exit,” and the like.

Situation status component 1712 may actively inquire about suchsituational status information (e.g., transmit requests forinformation), may passively remain open to receiving any suchinformation that is transmitted by a unit (e.g., a receiver configuredto receive signal transmissions from units), or both.

Priority component 1714 may determine a priority for one or more targetterminals 1100 among a plurality of target terminals 100. Priority maybe determined by applying one or more predefined priority criteria, suchpriority criteria based upon one or more of user location information,equipment location information, registered user detail information, andsituational status information. Priority criteria may be defined in anymanner. For example, priority criteria may include an expression thatcomputes a weighted score for each target terminal under consideration,and rank the imminence of the need to assist the targets based on thescore. For example, a priority criteria may be given by the expression:

PRIORITY SCORE=(Factor1*w1)+(Factor2*w2)+ . . . +(FactorN*wN)   [1]

For example, suppose that an implementation of system 1000 considers twodifferent factors in determining priority. Suppose, in this example,that Factor1 represents a score based on the time within whichassistance must be rendered, and Factor2 represents a score based onwhether there are children in the vicinity of a responder or not. Suchscores and/or scoring criteria may be preset or otherwise predefined andstored within system 1000.

For example, suppose the following scores are associated with thefollowing times within which assistance must be rendered (e.g., forFactor1):

TABLE 1.0 Time ranges Score  0 min ≤ t ≤ 5 min 5  5 min < t ≤ 10 min 410 min < t ≤ 20 min 3 20 min < t ≤ 30 min 2 30 min < t ≤ 60 min 1 60 min< t 0Further suppose that the following messages (column B of Table 1.1) arereceived at particular time (column D of Table 1.1) from three targetterminals (TU) (column A of Table 1.1), and the following scores (columnF) are determined for each based on, in this example, informationgathered from the message itself (e.g., “AT” in column C of Table 1.1.),the time the message was received (“TR” in column D of Table 1.1), andthe present time (CT in Table 1.1). In this particular example, thescoring expression may be given by (RT+t)−CT, and the score may bedetermined by a comparison of the result of the scoring expression withTable 1.0 above.

TABLE 1.1 Current Time (CT) = 12:00 PM C D Estimated time within TimeTBM F B which assistance is to was received E Factor1 A Message berendered (AT) (TR) (TR + 1) − CT Score TU 1 “A child in my vicinity 1minute or less 12:00 PM (12:00PM + 0:01) − 5 is unconscious and needs12:00PM = 0:01 help immediately” 1 minute TU 2 “I am trapped alone in 2hours 12:02 PM (12:02PM + 2:00) − 0 a room where the rubble 12:00PM =2:02 from the earthquake has 122 minutes blocked the door; but I am safeand could wait for up to 2 hours for assistance if others need helpfirst” TU 3 “I sustained an injury 20 minutes 11:58 AM (11:58AM + 0:20)− 3 and am bleeding pretty 12:00PM = 0:18 badly, I need assistance 18minutes within the next 20 minutes.

Extending the example above, suppose the following scores are based onwhether there are children in the vicinity of a responder or not (e.g.,Factor2):

TABLE 2.0 Children in the vicinity of messaging Target terminal Factor2Score Yes 2 Unknown 1 No 0

And further suppose, just as above with regard to Table 1.1, that thefollowing messages (repeated in column B of Table 2.1) are received atparticular time (repeated in column D of Table 2.1) from three targetterminals (TU) (repeated in column A of Table 1.1), and the followingscores for Factor2) (column F) are determined for each based on, in thisexample, information gathered from the message itself concerningchildren in the vicinity of the TUs. In this particular example, thescores may be determined by a comparison of a message analysis withTable 2.0 above.

TABLE 2.1 Current Time (CT) = 12:00 PM C D Estimate of whether Time TBME B any children are was received Factor 2 A Message within the vicinity(TR) Score TU 1 “A child in my vicinity Yes 12:00PM 2 is unconscious andneeds help immediately” TU 2 “I am trapped alone in No 12:02 PM 0 a roomwhere the rubble from the earthquake has blocked the door; but I am safeand could wait for up to 2 hours for assistance if others need helpfirst” TU 3 “I sustained an injury Unknown 11:58AM 1 and am bleedingpretty badly, I need assistance within the next 20 minutes.

System 1000 may apply the scoring expression given in Eqn. 1 above, forexample, and determine the priority scores accordingly. A summary ofsuch priority scores in the illustrative example above are shown inTable 3.0 below:

TABLE 3.0 21 Current Time (CT) = 12:00 PM C D E B Factor1 Factor2PRIORITY A Message Score Score SCORE TU 1 “A child in my vicinity 5 2 7is unconscious and needs help immediately” TU 2 “I am trapped alone in 00 0 a room where the rubble from the earthquake has blocked the door;but I am safe and could wait for up to 2 hours for assistance if othersneed help first” TU 3 “I sustained an injury 3 1 4 and am bleedingpretty badly, I need assistance within the next 20 minutes.

Based on the priority scores computed by system 1000 and shown above inTable 3.0, system 1000 may determine that, as between the three targetterminals in the above example, TU 1 is the first priority (with thehighest priority score), TU 3 is the second priority (with the secondhighest priority score), and TU 2 is the third priority (with the lowestpriority score). System 1000 may allocate assignments for emergencyresponders to be dispatched to the aid of the target terminals inaccordance with the priority determined among the plurality of unitsunder consideration for a given application.

Mission assignment component 1716 matches candidate emergency responderswith targets in need of assistance, and may issue a notification orother message to the emergency responder through an emergency responderterminal 1200, and/or to the target through a target terminal 1200.Mission assignment component 1716 may determine which one or moreemergency responders should be dispatched to assist one or more targetsin need of assistance, and such determination may be determined on anyone or more of: a location of a candidate emergency responder terminal1100, a location of a target terminal 1200 in need of assistance, alocation of a piece of equipment, a priority among a plurality oftargets, one or more pieces of information stored in a user profile forone or more registered users (either or both of emergency responders ortargets associated with emergency responder terminals 1100 and targetterminals 200), or any information derived from the foregoing (e.g.,absolute distance or route distance between a given emergency responderterminal and a given target terminal). Mission assignment component 1716may, in accordance with one or more embodiments, further receivefeedback from a registered user (e.g., an emergency responder, a target,etc.) indicating the need for additional assistance for a particularassignment, or the need for a reassignment for one reason or another.System 1000 may automatically make an additional assignment (e.g.,summon another emergency responder), make a new assignment, or modify anassignment based on the feedback and further based on other informationavailable to it. The other information available to System 1000 andwhich may be used for determining an additional assignment, newassignment, or modification to an assignment may include: (i) nearbyemergency responders available to assist, (ii) nearby emergencyresponders with a given skillset or with a given capability, (iii)nearby target users able to assist, etc. An example implementation isdiscussed with reference to selectable icon 1953 and selectable icon1954 of FIG. 3C.

Mission route component 1718 may determine one or more routes providingpossible paths of travel that an emergency responder may follow in orderto reach the target terminal 1200. Mission route component 1718 may drawupon map information stored within system 1000 (e.g., within electronicstorage 1730) or otherwise accessible to system 1000 (e.g., via a mapdatabase accessible online) to determine and/or provide a route uponwhich the responder may travel to reach the target terminal 1200. Suchmap information may be based on building information (e.g., stairwelllocations, elevator bays, escalators), online map information (e.g.,google maps information). The route provided may be a walking route,driving route, bicycle route, or any route suitable for any mode oftravel (escalator route, elevator route, etc.) or any combination of theforegoing. The determined route or routes may be displayed as an overlayon a portion of a real-world scene captured by a camera component 1706of an emergency responder terminal, or may be provided as a list ofvisual instructions, a list of audio instructions, or in any otherformat or manner desired. In some embodiments, multiple routes aredetermined and provided to an emergency responder via the emergencyresponder's emergency responder terminal 1200.

Referring back now to display object component 1708, in some embodimentsa display object component 1708 may define one or more display objectsrepresenting a route of travel depicting a virtual path between thelocation of an emergency responder terminal 1100 and the location of atarget terminal 1200, location data for which having been obtainedand/or stored by system resources 1800, and the route determined bymission route component 1718. A display object may include any graphic(e.g., a broken or solid line, arrow, etc.) that may be displayed on anelectronic display of the emergency responder terminal.

Display object component 1708 may dynamically refresh and/or adaptdisplay object definitions such that the display objects displayed ondisplay of respective units update to reflect recent information (e.g.,location or orientation of respective units). For example, displayobject component 1708 may adjust the position of display objectsrepresenting a route responsive to a determination that a portion of areal-world scene captured by a camera component 1706 of an emergencyresponder terminal 1100 has changed; the changed portion capturing adifferent region of the real-world scene than was previously captured,such that the region of space where the target terminal 1200 location isrepresented on the display changes.

Equipment location component 1722 in accordance with one or moreembodiments of the present disclosure obtains equipment location data ofone or more pieces of equipment in the vicinity of the emergency (thevicinity defined as desired for a given implementation). In someembodiments, such equipment location data may be obtained via locationresources (e.g., location circuitry) local to or coupled with pieces ofsuch equipment, and may be provided to system 1000 over network 1600(which in some instances may be include mesh network options). Equipmentlocation data is indicative of the geospatial location of one or morepieces of equipment in the vicinity of the emergency.

Equipment location component 1722 may include components coupled with ornear equipment (i.e., equipment local resources), components included inemergency management entity (e.g., communications modules,machine-readable instructions, etc.), or both. In accordance with one ormore embodiments of the present disclosure, system 1000 may obtainlocation data by actively transmitting a location data request to one ormore location resources coupled with or near pieces of equipment,thereby prompting the one or more location resources (e.g., GPS modules,altimeter, barometer, beacons, RFID tags, etc.) to obtain locationinformation and cause a data packet containing the location data (e.g.,location details) to be generated and/or transmitted to a computingresource of system 1000. For instance, EME 1300 of system 1000 mayreceive a responsive transmission from the one or more pieces ofequipment (i.e., from a communication interface coupled with a piece ofequipment) containing the one or more pieces of equipment's respectiveequipment location data, and system 1000 may propagate and/or store suchinformation within an electronic storage or memory of system 1000 asdesired for a particular application or by a particular resource coupledto or within system 1000.

Equipment location data may indicate geospatial location of a piece ofequipment in the vicinity of the emergency, including longitude andlatitude coordinates, degrees/minutes/seconds location parameters,altitude above sea level, altitude above ground level, etc. Equipmentlocation component 1722 may be utilized to identify geospatial locationof a piece of equipment. Equipment location component 1722 may compriseone or more circuits, modules, or chips local to the units themselves.For example, equipment location component 1722 may include a GPS sensor,an altimeter, a pressure sensor (e.g., a barometer), beacon (e.g.,Bluetooth beacon), RFID tag, and the like. In some embodiments,equipment location component 1722 may further comprise hardware andsoftware operating on EME 1300 and communicatively coupled with locationsensors of one or more units.

FIG. 3A depicts an example emergency responder 1912 utilizing thepresently disclosed technology in an example environment in which thepresently disclosed technology may be implemented. As shown, emergencyresponder 1912 is holding up his emergency responder terminal 1920 tocapture a portion of the real-world scene he is within (the cameracomponent of emergency responder terminal 1920 being pointed in thedirection of the portion of the real-world scene the emergency responderdesires to capture). Here, the portion of the real-world scene may be acity block intersection that includes, for example, buildings 1902,1904, automobiles 1906, 1907, city streets 1908, 1910, city benches1914, street lighting, plant life, pedestrians (which may or may not beother registered users), etc.

FIG. 3B illustrates an example augmented reality presentation that maybe displayed on the emergency responder 1912's emergency responderterminal 1920 of FIG. 3A. As shown, the presentation may include one ormore augmented reality display objects corresponding to registered userlocations (e.g., target location display objects 1930, 1931, 1932, 1934,1935, 1936, 1940; emergency responder location display object 1937) inaccordance with one or more embodiments of the present disclosure. Thatis, display objects 1930-1940 are provided as an augmented realityoverlay to indicate where in the real-world scene other registered userswithin the scene being captured are located. Because user locationcomponent 1704 of system 1000 may determine users' altitude as well aslongitudinal and latitudinal coordinates, display objects 1930-1940 maybe positioned to reflect both horizontal and vertical displacement asbetween the emergency responder terminal 1920 and the other terminalswithin the real-world scene being captured. In some embodiments, thedisplay objects may show horizontal distance, vertical distance, orother information as part of the display object itself (shown in FIG.4). In some embodiments, a visual feature (e.g., size, color, shape,etc.) of a display object may be used to convey relative distanceinformation about user locations relative to the given emergencyresponder's location. For example, although not shown in FIGS. 3A-3G forsimplicity, display objects associated with user terminals whoselocation is further from the emergency responder's terminal than othersmay appear to be smaller; and likewise display objects associated withuser terminals whose location is closer to the emergency responderterminal than others may appear to be bigger. Thus, in some embodimentsthe size (or other visual feature) of a display object being displayedon an emergency responder's terminal may be displayed having a size thatis a function of the distance between the terminal associated with thedisplay object and the emergency responder's terminal (shown in FIG. 4).In some embodiments, display objects may be selectable to drill downinto other information about the user (e.g., location details,registered user data), and/or options associated with the given user, asillustrated in FIG. 3C.

FIG. 3C illustrates example registered user information 1942 and examplerelated options 1950 displayed in an augmented reality presentationsimilar to that shown in FIG. 3B. The registered user information 1942and related options 1950 are associated with the user that correspondsto display object 1940 in the presentation. As shown in this example,such registered user information 1942 and related options 1950 may bemay be presented in an example pop-up field 1941. In some embodiments,registered user information 1942 and/or related options 1950 may appearin such an example pop-up field 1941 upon user selection, for example,such as the emergency responder 1912 tapping the display object 1940 onthe touchscreen display of their emergency responder terminal 1920 toreveal the additional information and options. In some embodiments,registered user information 1942 and/or related options 1950 may appearin such an example pop-up field 1941 upon a mission assignment from anemergency supervisor, for example, such as an emergency supervisoroverseeing a team of emergency responders making a selection that issuesa command that assigns emergency responder 1912 to be dispatched to aidthe registered user associated with display object 1940. Upon theassignment being made, the additional registered user information andother options may be automatically presented. In such embodiments, asupervisor or other manager of a team of emergency responders canquickly and efficiently match and dispatch certain of their emergencyresponders to go render assistance to certain of the registered users inneed of assistance (i.e., targets).

As shown, example registered user information provided in pop-up field1941 may include the horizontal distance and/or vertical distance fromone or more other points in space. In FIG. 3C, as may be seen, there isa horizontal distance of approximately 0.2 miles between the emergencyresponder 1912's terminal and the target terminal associated with thedisplay object 1940. There is a vertical distance of approximately 45feet between the emergency responder 1912's terminal and the targetterminal associated with the display object 1940.

In some embodiments, the distances presented to the user may reflectdifferent measures. For instance, the point in space from which tocompute the vertical distance might be ground level, sea level, theheight/altitude of the emergency responder (or other registered user),or any other point as desired for the given application; and the pointin space from which to measure the horizontal distance might be thelatitudinal and longitudinal coordinates of the emergency responder (orother registered user), or any other point as desired for the givenlocation. In some embodiments, one or more of the distances providedrepresent the shortest horizontal and shortest vertical distancesbetween the point in space and the location of the registered user inspace (e.g., a straight horizontal line between the emergency responder1912's location and the target's location (represented by display object1940). In some embodiments, one or more of the distances provided mayrepresent the horizontal and/or vertical distance that would need to betraversed to follow a recommended route between the two points (such aroute is shown and discussed in more detail with respect to FIG. 3D). Insome embodiments, a distance measure provided may be a line-of-sightdistance (that is, the length of the hypotenuse that is opposite theright angle formed between a straight line representing the shortesthorizontal distance and straight line representing the shortest verticaldistance line).

In some embodiments, the system may provide a route distance if routeinformation can be obtained (e.g., via a maps application, based onbuilding structure information, etc.), and alternatively a shortesthorizontal and/or shortest vertical distance and/or a line-of-sightdistance if no route information can be obtained (or the routeinformation is inadequate or unselected). One of ordinary skill in theart will appreciate that any one or more units of distance measure(e.g., feet, meters, inches, yards, miles, kilometers, etc.) may be usedin connection with the present disclosure.

As further shown in pop-up field 1941, registered user information mayinclude, by way of example and not by way of limitation, the user'sname, physical characteristics, medical or health conditions, skillsets,or any other information obtained by or accessible to system 1000. Thisinformation may be helpful for an emergency responder (or the emergencyresponder's supervisor) to better understand how to help the person inneed, and/or to more appropriately dispatch emergency responders to theaid of a given person. For example, the registered user associated withdisplay object 1940 in FIG. 3C, named Marlene Marlow, is registered asbeing 5′4″ tall and 320 pounds, and has medical conditions includingasthma and a condition that constrains her to a wheelchair. Thus, theemergency responder 1912 may take extra steps to prepare to helpMarlene.

These extra steps may include, by way of example and not by way oflimitation, texting or calling Marlene to establish contact and/or todetermine if Marlene's wheelchair is operable and/or if she hasknowledge of an accessible wheelchair ramp in the building, for example(e.g., by pressing icon 1951 for texting, or by pressing icon 1952 forvoice or video calling, etc.). These extra steps may include summoninganother emergency responder for additional help, for example, if theemergency responder 1912 recognizes that more than one person will berequired to lift Marlene out of the building (e.g., by pressing icon1953 to summon another nearby emergency responder, or other registereduser with a skillset that could be employed to assist Marlene). Theseextra steps may include transmitting a request to re-assign the missionto assist Marlene to another emergency responder with a differentskillset (assuming the emergency responder was assigned the mission inthe first place). For instance, emergency responder 1912 may not haveany equipment (such as an inhaler) to help a person having an asthmaattack (assuming this were the case), and may press icon 1954 to requesta re-assignment of the mission to another emergency responder that doeshave such equipment. Such requests may be pre-programmed and/orselectable from a menu, or may be customized messages sent throughin-app messaging or SMS messaging between emergency responders. In someembodiments, certain registered user information and/or options may bedisplayed as part of the original display object itself such that theuser need not select anything to see such certain registered userinformation about a particular user. Such certain information mayinclude any information desired that is accessible to or stored bysystem 1000.

In some embodiments, an emergency responder 1912 may be provided withroute information providing a route path to arrive at the location ofthe target. Such route information may be in the form of a list ofdirections, or a graphic representation of the route path. For example,the emergency responder 1912 in the present example may press button1955 to show a graphical representation of a route path between theemergency responder and the target as shown in FIG. 3D. As shown in FIG.3D, the route path 1965 may be displayed in the presentation to show thepath the emergency responder may follow to arrive at the location of thetarget. FIG. 3D shows icon 1955 shaded to reflect that it has beenselected. The route path may dynamically adjust as the location ofeither the emergency responder or the location of the target changes(based on the location of their respective terminals). The route pathmay dynamically adjust to reflect new information obtained by system1000 about possible routes. For example, system 1000 may receive anindication from a first emergency responder that a particular stairwellis blocked by an active fire, so the only other route is to access theupper floors is by the fire-escape ladder on the east side of thebuilding. Accordingly, system 1000 may route or re-route and display anew route path that reflects the updated information and informs theemergency responder that they must proceed to a different entry pointthan might otherwise be expected.

Although not shown in the augmented reality presentation of FIG. 3D,other information related to the route may be displayed in connectionwith the route path, such as, for example, an estimated time of arrival(based on average walking/running speeds, based on the particularemergency responder's walking/running speeds), an estimated time withinwhich an emergency responder must arrive (e.g., help NEEDED within 5minutes) before the situation becomes critical/fatal. The routeinformation may further include mission recommendations such as, run 0.2miles along path to building, walk to east side of building, climbladder, break the seventh window you arrive at while climbing theladder, proceed into the building with caution, press icon to activatean alarm on the target's terminal, proceed to sound, and so on. Suchmission recommendations may be based on one or more of (i) the timewithin which an emergency responder must arrive to render adequate care,(ii) the given emergency responder's location relative to the target'slocation, (iii) the route path, (iv) the emergency responder's physicalcapabilities (based on historical trends, inputs from the emergencyresponder, etc.), (v) the tools the emergency responder has with them,(vi) the building structure as known to system 1000, (vii) feedback fromother emergency responders providing updates about the building,compromised routes of travel, or feedback about the emergency situationgenerally.

In some embodiments, an emergency responder 1912 may be provided withuser path (e.g., target path) information providing a user path, or aportion of a user path, that led them to their current location. Suchinformation may be useful to identify how a particular user arrived attheir current location, such information further being useful toidentify a possible path to follow to get to the target location, toidentify users that may have caused or seen what caused some action thatgave rise to the emergency situation, among other information. Such userpath information be shown as a graphic delineating a path. For example,after pressing display object 1940, emergency responder 1912 may pressbutton 1956 to show a graphical representation of the user path 1966 afor the target user associated with the display object 1940. As shown,user path 1966 a in the example provided in FIG. 3E is shown by a seriesof black dots that convey the user's path. Although not shown in FIG.3E, some graphical representations of a user path may indicate thedirection of the user's travel. For instance, considering FIG. 3E, ifthe user associated with display object 1940 traveled from the bench1914 to the second floor of building 1904, the dots between the bench1914 and the building 1904 may include an arrow or other pointerpointing substantially in a leftward direction. As shown in FIG. 3E, theuser path 1966 may be displayed in the presentation to show the path agiven target travelled to arrive at their current location. FIG. 3Eshows icon 1956 shaded to reflect that it has been selected.

In some embodiments, as shown by way of example in FIG. 3F, multipleuser paths for multiple users may be displayed in the same presentation.For instance, in addition to user path 1966 a (denoted by black dots),the presentation may further show user path 1966 b (denoted by dotshaving a horizontal line pattern) that shows the path that the userassociated with display object 1938 followed to arrive at their currentlocation. In some such embodiments, one or more user paths may partiallyor entirely overlap. In some such embodiments, the graphicalrepresentation provided for the overlapping portion of multiple users'paths may be adjusted to exhibit a visual feature denoting an increaseddensity of individuals traveling along the same path (e.g., to denote aheavily traveled path). For example, instead of showing multiple linesof dots on top of one another or immediately next to one another, asingle line of dots may be shown as larger dots in areas of overlap.That is, the size of the graphical representation used to denote userpaths may be adjusted to reflect increased or decrease traffic alongsuch paths, or portions of such paths. In some instances, the areas ofoverlap are distinguished by visual features other than, or in additionto, size. For instance, as shown in FIG. 3F, the portion of the userpaths that overlap is denoted with dots having a checker pattern fillrather than a solid fill (as user path 1966 a) or a horizontal linepattern fill (as user path 1966 b).

As one of ordinary skill in the art will appreciate upon review of theinstant disclosure, any graphical representation desired maybe use todelineate a user path or a route path. The dots and lines used in FIGS.3D-3G are in no way intended to be limiting, but instead are merely usedas illustrative aids to help describe features presented in the instantdisclosure.

FIG. 3G illustrates the example augmented reality presentation shown inFIG. 3E, here further showing example equipment locations in accordancewith one or more embodiments of the present disclosure. As noted, insome embodiments, system 1000 may obtain location information of one ormore pieces of equipment within (or associated with) the zone ofinterest, and may further provide such location information of one ormore pieces of equipment to the one or more emergency responderterminals and/or present an augmented reality display object on thedisplay of such emergency responder terminals. Such equipment mayinclude a fire extinguisher, an epinephrine pen (EpiPen), an axe, anoxygen mask, a defibrillator, a hose, a mask (e.g., an oxygen mask, anantimicrobial mask), a fire alarm, a fire hydrant, a hose, a rope, etc.For example, as shown in the example illustration in FIG. 3F, displayobjects (represented collectively by numeral 1967 a) corresponding tofire extinguishers are presented as augmented reality overlays atop thepresented image in locations where one or more pieces of equipment existwithin the real-world scene being presented. In another example, asshown, display object 1967 b corresponding to a defibrillator ispresented as augmented reality overlays atop the presented image in alocation where such defibrillator exists within the real-world scenebeing presented. In another example, as shown, display object 1967 ccorresponding to a package of antimicrobial masks is presented as anaugmented reality overlay atop the presented image in a location wheresuch package of antimicrobial masks exists within the real-world scenebeing presented (here, on the top floor of building 1904). One ofordinary skill in the art will appreciate that any display objectdesired may be presented to correspond to a location of any piece ofequipment relevant to a particular emergency situation. Differentbuildings may have different equipment. Thus, in differentimplementations of system 1000, different icons may be associated withsuch equipment and presented as augmented reality overlays as a user(e.g., emergency responder 1912) views a portion of a scene as capturedvia the camera component of their terminal. In some embodiments,equipment locations may appear within the augmented reality presentationupon user selection, for example, a user's selection of equipment icon1957 shown in related options field 1950.

FIG. 4 illustrates another example augmented reality presentation thatmay be displayed on an emergency responder's emergency responderterminal in accordance with one or more embodiments of the presentdisclosure. The presentation may include display objects associated withusers in different locations within building 1970. As shown, the displayobjects themselves may include one or more details about the registeredusers associated with them. For example, display objects 1971-1975 mayinclude a distance measure and an altitude measure for individual users.Additionally, the size of the display objects may be keyed to thedistance measure. For instance, the user associated with display object1971 is 0.20 miles from the emergency responder holding the terminalshown, and the user associated with display object 1975 is 0.22 milesfrom the emergency responder holding the terminal shown. Since the userassociated with display object 1971 is closer to the emergency responderholding the terminal shown than the user associated with display object1975, display object 1971 is larger than display object 1975. Thus, theaugmented reality display objects may exhibit a visual feature thatprovides depth cues to indicate to the emergency responder how close agiven user is relative to other users (i.e., user associated withdisplay object 1971 is closer than user associated with display object1975). Similar visual features may be employed with respect to displayobjects associated with equipment. Further as shown in FIG. 4, thepresentation may include a compass indication display object 1976 and/oraltitude indication 1977 based on the position and/or orientation of theterminal shown.

FIG. 5 is a flow diagram illustrating an example method 2000 that may beimplemented by system 1000 in accordance with one or more embodiments ofthe present disclosure. As shown, at operation 2002 method 2000 mayobtain location data associated with one or more of a target terminaland an emergency response terminal. At operation 2004, method 2800 maydetermine a horizontal distance measure associated with the targetterminal, based on the location data. At operation 2006, method 2000determine a vertical distance measure associated with the targetterminal, based on the location data. At operation 2008, method 2000define a selectable display object associated with the location data. Atoperation 2010, method 2000 may engage a camera component to capture oneor more images of at least a portion of the real-world scene surroundingthe emergency response terminal, and provide in real-time the one ormore images for display on the touchscreen display of the emergencyresponse terminal. At operation 2012, method 2000 may provide theselectable display object for display on the touchscreen of theemergency response terminal when the image provided for display spans aregion of the real-world scene associated with the location data of thetarget terminal, the selectable display object provided for display in aposition on the touchscreen such that it overlays a segment of thedisplayed image associated with the region. At operation 2014, method2000 may obtain a mission assignment, the mission assignment based onone or more of: registered user details associated with the user of thetarget terminal, registered user details associated with the user of theemergency responder terminal, and a priority score of the targetterminal among a plurality of target terminals.

FIG. 6 is a flow diagram illustrating another example method 2100 thatmay be implemented by system 1000 in accordance with one or moreembodiments of the present disclosure. As shown, at operation 2102method 2000 may obtain, generate, determine, define and/or providelocation data associated with one or more of a target terminal and anemergency response terminal, the location data including one or more of:a latitude position measure, a longitude position measure, and avertical position measure. At operation 2104, method 2000 may obtain,generate, determine, define and/or provide a horizontal distance measureassociated with the target terminal, the horizontal distance measurebased on one or more of the latitude position measure, the longitudeposition measure, and the vertical position measure. At operation 2106,method 2100 may obtain, generate, determine, define and/or provide avertical distance measure associated with the target terminal, thehorizontal distance measure based on one or more of the latitudeposition measure, the longitude position measure, and the verticalposition measure. At operation 2108, method 2100 may obtain, generate,determine, define and/or provide a selectable display object associatedwith the location data associated with the target terminal. At operation2110, method 2100 may engage a camera component to capture one or moreimages of at least a portion of the real-world scene surrounding theemergency response terminal, and obtain, generate, determine, define orprovide in real-time the one or more images for display on thetouchscreen display of the emergency response terminal. At operation2112, method 2100 may obtain, generate, determine, define and/or providethe selectable display object for display on the touchscreen of theemergency response terminal when the image provided for display spans aregion of the real-world scene associated with an estimated real-worldlocation of the target terminal, the selectable display object providedfor display in a position on the touchscreen such that it overlays asegment of the displayed image associated with the region. At operation2114, method 2100 may obtain, generate, determine, define and/or providea mission assignment, the mission assignment based on one or more of:registered user details associated with the user of the target terminal,registered user details associated with the user of the emergencyresponder terminal, and a priority score of the target terminal among aplurality of target terminals.

Referring to FIGS. 1-7 collectively, although these illustrate exampleembodiments with components, elements and circuits partitioned in thedepicted manner, it will be appreciated by one of ordinary skill in theart that various components and circuits of system 1000 (including anyone or more elements and subsystems related thereto, individually orcollectively) and described herein may be implemented utilizing any formof hardware, software, or a combination thereof. For example, one ormore processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logicalcomponents, software routines or other mechanisms, including associatedmemory, might be used to implement one or more components or circuits inembodiments of system 1000 (including any one or more elements andsubsystems related thereto) described in the present disclosure. Inembodiments, the various components and circuits described herein mightbe implemented as discrete components or the functions and featuresdescribed can be shared in part or in total among two or morecomponents. In other words, as would be apparent to one of ordinaryskill in the art after reading this description, the various featuresand functionality described herein may be implemented in any givenapplication and can be implemented in one or more separate or sharedcomponents in various combinations and permutations. Even though variousfeatures or elements of functionality may be individually described orclaimed as separate components, in various embodiments these featuresand functionality can be shared among one or more common software andhardware elements, and such description shall not require or imply thatseparate hardware or software components are used to implement suchfeatures or functionality.

As used herein, the term “component” might describe a given unit offunctionality that can be performed in accordance with one or moreembodiments of the technology disclosed herein. As used herein, acomponent might describe a given unit of hardware that can be utilized,alone or in conjunction with software, to perform an operation (e.g., acamera component might include a CMOS array, an analog-to-digitalconverter, a sampling circuit, and software to create a permanent ortemporary digital image information, for instance). Thus, as usedherein, a component might comprise or be implemented utilizing any formof hardware, software, or a combination thereof. For example, one ormore processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logicalcomponents, software routines or other mechanisms might be implementedto make up a component. In implementation, the various componentsdescribed herein might be implemented as discrete components, or thefunctions and features described can be shared in part or in total amongone or more components. In other words, as would be apparent to one ofordinary skill in the art after reading this description, the variousfeatures and functionality described herein may be implemented in anygiven application and can be implemented in one or more separate orshared components in various combinations and permutations. Even thoughvarious features or elements of functionality may be individuallydescribed or claimed as separate components, one of ordinary skill inthe art will understand that these features and functionality can beshared among one or more common software and hardware elements, and suchdescription shall not require or imply that separate hardware orsoftware components are used to implement such features orfunctionality.

Where components of the technology are implemented in whole or in partusing software, in one embodiment, these software elements can beimplemented to operate with a computing or processing component orengine capable of carrying out the functionality described with respectthereto. One such example computing component is shown in FIG. 7.Various embodiments are described in terms of this example-computingcomponent 3200. After reading this description, it will become apparentto a person skilled in the relevant art how to implement the technologyusing other computing modules or architectures.

Referring now to FIG. 7, computing component 3200 may represent, forexample, computing or processing capabilities found within desktop,laptop and notebook computers; hand-held computing devices (PDA's, smartphones, cell phones, palmtops, etc.); mainframes, supercomputers,workstations or servers; or any other type of special-purpose orgeneral-purpose computing devices as may be desirable or appropriate fora given application or environment. Computing component 3200 might alsorepresent computing capabilities embedded within or otherwise availableto a given device. For example, a computing component might be found inother electronic devices such as, for example, digital cameras,navigation systems, cellular telephones, portable computing devices,modems, routers, WAPs, terminals and other electronic devices that mightinclude some form of processing capability.

Computing component 3200 might include, for example, one or moreprocessors, controllers, control modules, or other processing devices,such as a processor 3204. Processor 3204 might be implemented using ageneral-purpose or special-purpose processing engine such as, forexample, a microprocessor, controller, or other control logic. In theillustrated example, processor 3204 is connected to a bus 3202, althoughany communication medium can be used to facilitate interaction withother components of computing module 3200 or to communicate externally.

Computing component 3200 might also include one or more memory modules,simply referred to herein as main memory 3208. For example, preferablyrandom access memory (RAM) or other dynamic memory, might be used forstoring information and instructions to be executed by processor 3204.Main memory 3208 might also be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 3204. Computing component 3200 might likewiseinclude a read only memory (“ROM”) or other static storage devicecoupled to bus 3202 for storing static information and instructions forprocessor 3204.

The computing component 3200 might also include one or more variousforms of information storage mechanism 3210, which might include, forexample, a media drive 3212 and a storage unit interface 3220. The mediadrive 3212 might include a drive or other mechanism to support fixed orremovable storage media 3214. For example, a hard disk drive, a floppydisk drive, a magnetic tape drive, an optical disk drive, a CD or DVDdrive (R or RW), or other removable or fixed media drive might beprovided. Accordingly, storage media 3214 might include, for example, ahard disk, a floppy disk, magnetic tape, cartridge, optical disk, a CDor DVD, or other fixed or removable medium that is read by, written toor accessed by media drive 3212. As these examples illustrate, thestorage media 3214 can include a computer usable storage medium havingstored therein computer software or data.

In alternative embodiments, information storage mechanism 3210 mightinclude other similar instrumentalities for allowing computer programsor other instructions or data to be loaded into computing component3200. Such instrumentalities might include, for example, a fixed orremovable storage unit 3222 and an interface 3220. Examples of suchstorage units 3222 and interfaces 3220 can include a program cartridgeand cartridge interface, a removable memory (for example, a flash memoryor other removable memory module) and memory slot, a PCMCIA slot andcard, and other fixed or removable storage units 3222 and interfaces3220 that allow software and data to be transferred from the storageunit 3222 to computing component 3200.

Computing component 3200 might also include a communications interface3224 (e.g., communications component 1722 shown in FIG. 2).Communications interface 3224 might be used to allow software and datato be transferred between computing component 3200 and external devices.Examples of communications interface 3224 might include a modem orsoftmodem, a network interface (such as an Ethernet, network interfacecard, WiMedia, IEEE 802.XX or other interface), a communications port(such as for example, a USB port, IR port, RS232 port Bluetooth®interface, or other port), or other communications interface. Softwareand data transferred via communications interface 3224 might typicallybe carried on signals, which can be electronic, electromagnetic (whichincludes optical) or other signals capable of being exchanged by a givencommunications interface 3224. These signals might be provided tocommunications interface 3224 via a channel 3228. This channel 3228might carry signals and might be implemented using a wired or wirelesscommunication medium. Some examples of a channel might include a phoneline, a cellular link, an RF link, an optical link, a network interface,a local or wide area network, and other wired or wireless communicationschannels.

In this document, the terms “computer program medium” and “computerusable medium” are used to generally refer to media such as, forexample, memory 3208, storage unit 3220, media 3214, and channel 3228.These and other various forms of computer program media or computerusable media may be involved in carrying one or more sequences of one ormore instructions to a processing device for execution. Suchinstructions embodied on the medium, are generally referred to as“computer program code” or a “computer program product” (which may begrouped in the form of computer programs or other groupings). Whenexecuted, such instructions might enable the computing component 3200 toperform features or functions of the disclosed technology as discussedherein.

While various embodiments of the disclosed technology have beendescribed above, it should be understood that they have been presentedby way of example only, and not of limitation. Likewise, the variousdiagrams may depict an example architectural or other configuration forthe disclosed technology, which is done to aid in understanding thefeatures and functionality that can be included in the disclosedtechnology. The disclosed technology is not restricted to theillustrated example architectures or configurations, but the desiredfeatures can be implemented using a variety of alternative architecturesand configurations. Indeed, it will be apparent to one of skill in theart how alternative functional, logical or physical partitioning andconfigurations can be implemented to implement the desired features ofthe technology disclosed herein. Also, a multitude of differentconstituent module names other than those depicted herein can be appliedto the various partitions. Additionally, with regard to flow diagrams,operational descriptions and method claims, the order in which the stepsare presented herein shall not mandate that various embodiments beimplemented to perform the recited functionality in the same orderunless the context dictates otherwise.

Although the disclosed technology is described above in terms of variousexemplary embodiments and implementations, it should be understood thatthe various features, aspects and functionality described in one or moreof the individual embodiments are not limited in their applicability tothe particular embodiment with which they are described, but instead canbe applied, alone or in various combinations, to one or more of theother embodiments of the disclosed technology, whether or not suchembodiments are described and whether or not such features are presentedas being a part of a described embodiment. Thus, the breadth and scopeof the technology disclosed herein should not be limited by any of theabove-described exemplary embodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; the terms “a” or“an” should be read as meaning “at least one,” “one or more” or thelike; and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “module” does not imply that the components or functionalitydescribed or claimed as part of the module are all configured in acommon package. Indeed, any or all of the various components of amodule, whether control logic or other components, can be combined in asingle package or separately maintained and can further be distributedin multiple groupings or packages or across multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives can be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

What is claimed is:
 1. An emergency response system (ERS) comprising: anon-transitory computer readable medium storing machine readableinstructions which when executed by a processor cause the ERS to: obtaintarget terminal location data for a plurality of target terminals, thetarget terminal location data including a latitude measure, a longitudemeasure, and an altitude measure, wherein each of the plurality oftarget terminals comprise a GPS circuit, an altimeter, and acommunications circuit; obtain emergency responder terminal locationdata for an emergency responder terminal, the emergency responderterminal location data including a latitude measure, a longitudemeasure, and an altitude measure, wherein the emergency responderterminal comprises a GPS circuit, an altimeter, a camera component, atouchscreen display, a communication circuit, a processing engine, and amemory; determine a horizontal distance measure between the emergencyresponder terminal in each of the plurality of target terminals, thehorizontal distance measure computed from the target terminal locationdata and the emergency responder terminal location data; determine avertical distance measure between the emergency responder terminal andeach of the plurality of target terminals, the vertical distance measurecomputed from the target terminal location data and the emergencyresponder terminal location data; engage the camera component of theemergency responder terminal to capture images of at least a portion ofa surrounding real-world scene, and provide such images for display onthe touchscreen display of the emergency responder terminal; provide atarget terminal display object for display on the touchscreen display ofthe emergency responder terminal such that, when an image provided fordisplay spans a region of the surrounding real-world scene within whicha target terminal is located, the target terminal display objectoverlays a segment of the image associated with the target terminallocation data, whereupon selection of the target terminal display objectcauses registered user details to be displayed on the touchscreendisplay, the registered user details including a physicalcharacteristic; provide one or more menu items associated with aselected target terminal display object, the one or more menu itemsconfigured to activate a resource of the target terminal of the selectedtarget terminal display object to assist the emergency responder tolocate the target terminal.
 2. The ERS of claim 1, wherein the resourceof the target terminal includes a speaker of the target terminal topropagate an alarm sound, a light source of the target terminal topropagate a flashing light, or a combination thereof.
 3. The ERS ofclaim 1, wherein the target terminal location data is obtained from oneor more of: components internal to a housing of the target terminal;components external to a housing of the target terminal; selection inputfrom a user of the target terminal; textual input from a user of thetarget terminal; voice input from a user of the target terminal; andvideo input from a user of the target terminal.
 4. The ERS of claim 3,wherein the components internal to the housing of the target terminalmay include one or more of a GPS sensor, an altimeter, and a pressuresensor.
 5. The ERS of claim 3, wherein the components external to thehousing of the target terminal include one or more target vicinitycomponents located in the vicinity of the target at a scene of anongoing emergency.
 6. The ERS of claim 3, wherein the componentsexternal to the housing of the target terminal include one or moretarget vicinity components located in the vicinity of the target at ascene of an ongoing emergency, wherein the target vicinity componentsinclude a plurality of proximity beacons configured to communicate withtarget terminals.
 7. The ERS of claim 1, wherein the non-transitorycomputer readable medium further stores machine readable instructionsthat, when executed by the processor, cause the ERS to: obtain equipmentlocation data from a plurality of equipment location components, theplurality of equipment location data including a latitude measure, alongitude measure, and an altitude measure, wherein the plurality ofequipment location components comprising one or more of a GPS sensor, analtimeter, a pressure sensor, a proximity beacon, or an RFID tag;determine a horizontal distance measure between the emergency responderterminal and each of the plurality of equipment location components, thehorizontal distance measure computed from the equipment location dataand the emergency responder terminal location data; determine a verticaldistance measure between the emergency responder terminal and each ofthe plurality of equipment location components, the vertical distancemeasure computed from the equipment location data and the emergencyresponder terminal location data; and provide an equipment locationdisplay object for each of the plurality of equipment locationcomponents for display on the touchscreen display of the emergencyresponder terminal such that, when an image provided for display spansthe region surrounding real-world scene within which a target terminalis located, each of the equipment location display objects overlay thesegment of the image associated with each equipment location component.8. The ERS of claim 7, wherein the plurality of equipment locationcomponents are associated with one or more of a fire extinguisher, anepinephrine pen, an axe, a mask, a defibrillator, a hose, a fire alarm,a fire hydrant, and a rope.
 9. The ERS of claim 7, wherein each of theequipment location display objects comprises an icon specific for acategory of equipment associated with each equipment location component.10. The ERS of claim 1, wherein the non-transitory computer readablemedium further stores machine readable instructions that, when executedby the processor, cause the ERS to: transmit a mission assignment to theemergency responder to render assistance to a specific target among theplurality of targets, the mission assignment based on: registered userdetails about the users of the plurality of target terminals, registereduser details about the user of the emergency responder terminal, andpriority scores among the plurality of target terminals; and provide amission route display object representing a suggested path of travelbetween the emergency responder terminal and the specific targetterminal.
 11. The ERS of claim 10, wherein the mission assignment isgenerated by an emergency management entity communicatively coupled withbut physically detached from the emergency responder terminal.
 12. Amethod comprising: obtaining target terminal location data for aplurality of target terminals, the target terminal location dataincluding a latitude measure, a longitude measure, and an altitudemeasure, wherein each of the plurality of target terminals comprise aGPS circuit, an altimeter, and a communications circuit; obtainingemergency responder terminal location data for an emergency responderterminal, the emergency responder terminal location data including alatitude measure, a longitude measure, and an altitude measure, whereinthe emergency responder terminal comprises a GPS circuit, an altimeter,a camera component, a touchscreen display, a communication circuit, aprocessing engine, and a memory; determining a horizontal distancemeasure between the emergency responder terminal in each of theplurality of target terminals, the horizontal distance measure computedfrom the target terminal location data and the emergency responderterminal location data; determining a vertical distance measure betweenthe emergency responder terminal and each of the plurality of targetterminals, the vertical distance measure computed from the targetterminal location data and the emergency responder terminal locationdata; engaging the camera component of the emergency responder terminalto capture images of at least a portion of a surrounding real-worldscene, and provide such images for display on the touchscreen display ofthe emergency responder terminal; providing a target terminal displayobject for display on the touchscreen display of the emergency responderterminal such that, when an image provided for display spans a region ofthe surrounding real-world scene within which a target terminal islocated, the target terminal display object overlays a segment of theimage associated with the target terminal location data, whereuponselection of the target terminal display object causes registered userdetails to be displayed on the touchscreen display, the registered userdetails including a physical characteristic; and providing one or moremenu items associated with a selected target terminal display object,the one or more menu items configured to activate a resource of thetarget terminal of the selected target terminal display object to assistthe emergency responder to locate the target terminal.
 13. The method ofclaim 12, wherein the resource of the target terminal includes a speakerof the target terminal to propagate an alarm sound, a light source ofthe target terminal to propagate a flashing light, or a combinationthereof.
 14. The method of claim 12, wherein the target terminallocation data is obtained from one or more of: components internal to ahousing of the target terminal; components external to a housing of thetarget terminal; selection input from a user of the target terminal;textual input from a user of the target terminal; voice input from auser of the target terminal; and video input from a user of the targetterminal.
 15. The method of claim 14, wherein the components internal tothe housing of the target terminal may include one or more of a GPSsensor, an altimeter, and a pressure sensor.
 16. The method of claim 14,wherein the components external to the housing of the target terminalinclude one or more target vicinity components located in the vicinityof the target at a scene of an ongoing emergency.
 17. The method ofclaim 12, further comprising: obtaining equipment location data from aplurality of equipment location components, the plurality of equipmentlocation data including a latitude measure, a longitude measure, and analtitude measure, wherein the plurality of equipment location componentscomprising one or more of a GPS sensor, an altimeter, a pressure sensor,a proximity beacon, or an RFID tag; determining a horizontal distancemeasure between the emergency responder terminal and each of theplurality of equipment location components, the horizontal distancemeasure computed from the equipment location data and the emergencyresponder terminal location data; determining a vertical distancemeasure between the emergency responder terminal and each of theplurality of equipment location components, the vertical distancemeasure computed from the equipment location data and the emergencyresponder terminal location data; and providing an equipment locationdisplay object for each of the plurality of equipment locationcomponents for display on the touchscreen display of the emergencyresponder terminal such that, when an image provided for display spansthe region surrounding real-world scene within which a target terminalis located, each of the equipment location display objects overlay thesegment of the image associated with each equipment location component.18. The method of claim 17, wherein the plurality of equipment locationcomponents are associated with one or more of a fire extinguisher, anepinephrine pen, an axe, a mask, a defibrillator, a hose, a fire alarm,a fire hydrant, and a rope.
 19. The method of claim 17, wherein each ofthe equipment location display objects comprises an icon specific for acategory of equipment associated with each equipment location component.20. The method of claim 12, further comprising: transmit a missionassignment to the emergency responder to render assistance to a specifictarget among the plurality of targets, the mission assignment based on:registered user details about the users of the plurality of targetterminals, registered user details about the user of the emergencyresponder terminal, and priority scores among the plurality of targetterminals; and provide a mission route display object representing asuggested path of travel between the emergency responder terminal andthe specific target terminal.